11. Normalizing Financial State Spaces

Part 1 - An Introduction to Normalization

Understanding Normalization in Data Processing

Normalization is an essential data pre-processing step that adjusts numeric feature values to a uniform scale without distorting value differences. It's crucial in AI to prevent any feature with a wider range from overshadowing others, ensuring:

• Equal Contribution: All features equally influence the learning model.
• Enhanced Model Efficiency: Results in better performance, accuracy, and training speed.
• Robust AI Systems: Offers reliability and stability in AI applications.

Techniques of Normalization

Two widely-used normalization methods are MinMax Scaling and Z Score Normalization:

• MinMax Scaling:

  • Transforms feature values to a range between 0 and 1.
  • Maintains relationships between data points.
  • Works well when feature bounds are known yet sensitive to outliers.

• Z Score Normalization:

  • Centers data at 0 with a standard deviation of 1.
  • Suitable for normally distributed data.
  • Less sensitive to outliers compared to MinMax.

Application Considerations

• MinMax: Best when data bounds are known.
• Z Score: Effective for data with unknown bounds and potential outliers.

Selecting the suitable normalization technique depends on the data and model requirements, aimed at minimizing output errors and enhancing performance.

Part 2 - Normalization of Financial Markets

Understanding Rolling Normalization in Financial Markets

Advantages of Rolling Normalization:

• Market Context Adaptation: Financial data is non-stationary, meaning its statistical properties change over time. Rolling normalization updates the min, max, mean, and standard deviation for each time window, reflecting current market conditions.
• Outlier Limitation: By focusing on smaller time frames, rolling normalization confines outliers to specific periods, preventing them from skewing long-term data interpretation.
• Data Relevance and Accuracy: Continuously updates in volatile markets, ensuring the data remains relevant by emphasizing recent observations.

Benefits for Learning Models:

• Increased Stability and Performance: Better captures financial data's dynamic nature, leading to improved model reliability.
• Prevention of Long-term Bias: Regularly updating parameters avoids biases from long-term shifts in data distribution.
• Facilitation of Incremental Learning: Supports models that evolve with new data, essential for real-time trading systems that must adapt quickly.

Conclusion:
Rolling normalization enhances learning models by adapting to changes, reducing outlier impacts, focusing on current data, and significantly improving trading decision accuracy.